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Abstract 

Reading comprehension can be considered a complex human performance involving two integrated repertoires: a verbal 
repertoire and an investigative (generative) repertoire. The analytical and reasoning skills necessary to demonstrate reading 
comprehension can be systematically taught by analyzing the verbal and investigative repertoires involved and then arrang¬ 
ing instructional contingencies designed to build and extend those repertoires. This paper describes the programing of such 
contingencies, drawing from the conceptual foundation outlined for reading comprehension (Layng, Sota, & Leon, 2011) and 
the analysis of the repertoires involved (Sota, Leon, & Layng, 2011). 
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I n discussing reading comprehension and introducing a theo¬ 
retical framework that guided the development of Headsprout 
Reading Comprehension , Layng, Sota, and Leon (2011) de¬ 
scribed a fictional scenario in which Jeffrey and John each an¬ 
swer a reading comprehension question. Jeffrey, who answers 
the question correctly, experiences private events that are dif¬ 
ferent from Johns. First, Jeffrey reads the passage, the question, 
and the possible answers. Next, he determines what the question 
is asking him to do: “It wants me to find how Sam feels. IT1 look 
back in the passage to see if I can find sad,’ ‘happy,’ or ‘funny.’” 
Then he looks back in the passage: “No... no... I can’t find any of 
those words in the passage. I’ll have to look for clue words that 
help me think about how Sam was feeling. Here it says, ‘When 
Sam saw his new bike, he grinned.’” Finally, Jeffrey answers the 
question: “Hmm... grinned.’ That makes me think Sam was feel¬ 
ing happy, and ‘happy’ is one of the possible answers. I’ll put a 
mark next to ‘happy.’” 

This scenario illustrates an instance of inferential com¬ 
prehension: part of the terminal repertoire that Head¬ 
sprout® Reading Comprehension was designed to build. 
Layng et al. (2011) argue that reading comprehension can 
be considered a complex human performance involving 
two integrated repertoires: a verbal repertoire and an in¬ 
vestigative repertoire. The above scenario illustrates two 
parts of an investigative repertoire applied to answering an 
inferential reading comprehension question: (1) an over¬ 
all strategy applied to answering multiple-choice reading 
comprehension questions and (2) a more specific strategy 
applied to answering inferential comprehension questions. 
It is important to note that different specific strategies are 
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required for each comprehension type— literal, inferen¬ 
tial, summative (main idea), and derived-word-meaning 
(vocabulary in context) comprehension. Together, these 
general and specific strategies make up the investigative 
repertoires taught in Headsprout Reading Comprehension. 

In addition to an investigative repertoire, a learner s ver¬ 
bal repertoire must be taken into consideration in the de¬ 
sign of any program of reading comprehension. The verbal 
repertoire is closely linked to the textual, intraverbal, auto¬ 
clitic, and tacting repertoires (see Skinner, 1957) needed to 
come into contact with key features of written text and is 
built over the course of a lifetime. A verbal repertoire can 
take much longer to establish than an investigative (strat¬ 
egy) repertoire. Nevertheless, it is an important part of 
reading comprehension; therefore, instructional activities 
designed to facilitate rapid acquisition of new vocabulary 
were built into the design of Headsprout Reading Compre¬ 
hension (see Sota, Leon, & Layng, 2011, for a more detailed 
discussion of factors considered in determining the target¬ 
ed terminal repertoire of the program). 

The current paper is the third in a series of three articles 
addressing the problem of teaching young learners how to 
comprehend written text. The three papers together pro¬ 
vide a somewhat detailed description of the design and 
development of Headsprout Reading Comprehension. The 
program comprises 50 online lessons lasting about 20 to 
30 minutes each. Once finished with the program, learners 
have a repertoire that enables them to comprehend writ¬ 
ten text, as indicated by their ability to answer questions 
about the text. The passages feature narrative, expository, 
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and poetic text. Science concepts, biography, measure¬ 
ment, using a table of contents, reading maps, following 
directions, and analyzing text using Venn and other types 
of diagrams are all incorporated into the program. 

The first article (Layng et al., 2011) described the theo¬ 
retical and procedural underpinnings of the effort. In that 
paper, the authors describe how thinking about text—and 
the private events that accompany thinking about text— 
may be approached within a contingency analytic frame¬ 
work and how that approach to private events informs the 
design of instruction. The heart of the approach requires 
the characterization of stimulus control topographies to 
include both dimensional aspects—what is responded 
to— and instructional/abstractional aspects—how one 
responds (see Goldiamond, 1966; Goldiamond & Thomp¬ 
son, 1967/2004). Accordingly, critical to the approach is 
defining the dimensional and instructional/abstractional 
relations important to both verbal and investigative rep¬ 
ertoires to be programed. That process is the topic of the 
second paper in the series (Sota et al., 2011). 

That task is not an easy one, as Sota et al. (2011) note: 

Comprehension is not a monolithic concept It is not 
something that someone either has or does not have. It is 
not something that someone either can do or cannot do. 
Reading comprehension is what we call it when particular 
responses are made in the presence of particular textual 
stimuli. Often, it refers to public events as well as private 
events which we would typically call thinking or reason¬ 
ing (see Layng, Sota, & Leon, 2011). When we say that a 
learner can comprehend what he or she has read, we are 
making a generalization statement based on a large pat¬ 
tern of stimulus-control topographies (Ray & Sidman, 
1970). These topographies vary across passages, questions, 
and responses. For example, reading material may vary in 
terms of the passages length, the reading level at which it is 
written, and its subject matter, as well as its style, sentence 
structure, vocabulary, and so on. A question about the pas¬ 
sage read may vary in terms of length, structure, and vo¬ 
cabulary, as well as the response required. Questions may 
be multiple-choice or open-ended. They may require a spo¬ 
ken response or a written one. These differences represent 
differences in stimulus-response relations and, ultimately, 
in the programing involved in building a reading compre¬ 
hension repertoire, (p. 11) 

Each element has to be analyzed and the dimensional and in¬ 
structional relations specified through a rigorous content analy¬ 
sis. Further, a hierarchy of relations must be defined and the 
constituent repertoires identified. Once the target relations are 
identified and, as a result, the criteria for assessing their occur¬ 
rence determined, a program needs to be designed and tested 
that takes a learner from basic decoding of text to comprehend¬ 
ing that text as defined by the content analysis. 

The current paper describes the programing process involved 
for establishing reading comprehension. This instructional de¬ 
sign process is informed by the approach described by Layng 


et al. (2011) and based on the analysis described by Sota et al. 
(2011). The following sections describe how Headsprout Read¬ 
ing Comprehension was designed to build learners verbal rep¬ 
ertoires and teach both general and specific investigative rep¬ 
ertoires. Whereas an exhaustive account of the process would 
likely require a textbook, we will attempt to illustrate the process 
by describing the program design and development specific to 
ensuring that learners verbal repertoires meet the programs 
vocabulary requirements (from context and directly taught), as 
well as the design specific to teaching inferential comprehen¬ 
sion. 

■ THE PROGRAM _ 

A program is a sequence of contingencies with changing criteria 
that leads to a replicable outcome. This outcome is (1) specified 
in advance, including both the repertoire that learners are ex¬ 
pected to demonstrate and the level of mastery they are expect¬ 
ed to reach, and (2) replicable, in that all learners who complete 
the program show similar terminal repertoires as evaluated by 
measures embedded within the program itself or administered 
after completion of the program. 

The outcome is achieved through a series of contingencies 
presented to the learner, each of which is changed slightly from 
the one before. Changes may span the three contingency terms, 
as previously neutral stimuli come to guide behavior, new re¬ 
sponses are shaped, and previously irrelevant consequences 
are potentiated. As the program advances, the requirements 
for reinforcement and progress are gradually raised. The first 
response requirement is preferably already part of the learner s 
repertoire, while the last response requirement is the terminal 
repertoire (see Twyman, Layng, Stikeleather, & Hobbins, 2004). 

■ PROGRAMING A VERBAL REPERTOIRE _ 

If what we call reading comprehension is to occur, there first 
needs to be a sufficient amount of overlap between the learn¬ 
ers’ verbal repertoire and the text of the passage, questions, and 
answers that learners read. The extent of this overlap may vary. 
If there is no overlap, the textual stimuli will fail to guide any 
verbal behavior on the part of the learner. If there is total over¬ 
lap, the writer is simply restating what is already in the readers 
repertoire. There must be enough overlap such that the learner s 
verbal responses to a text may be guided by stimulus-control 
relations similar to those that guide the writer or other members 
of the relevant verbal community, but there must be enough of 
a discrepancy such that new responses are encouraged. Accord¬ 
ingly, the program must provide instruction that increases the 
likelihood of such overlap, which can be illustrated by an ex¬ 
ample from our learning laboratory. 

ENSURING OVERLAP WITH LEARNERS’ VERBAL REPERTOIRES 

In initial iterations of the program, learners read a passage and 
saw a poster (shown in Figure 1) as part of their instruction on 
interpreting illustrations. Then, they answered questions about 
when the exhibit would be open. The questions were answered 
incorrectly by 4 out of 5 learners in our laboratory. Further, cor¬ 
rection procedures occasioned by an incorrect response had 
little corrective effect. 
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After that, we read fhaf the exhibit 
uill last from January through April 
I inallv. we read the small print when 
during the week the exhibit is open and 
how much it costs. 

When we go to an exhibit, we might 
see other posters or signs inside the 
building. The exhibit will be in the 
directum of the signs. If we aren't sure 


- 

Figure 1. Original text used in the passage, 





Cedarville’s 
Land and Sea Museum 

presents Antmsts of the Arctic. 

January 2nd through April 301 h 


Admission 

Tim., mudrw 



Alter that, wc read that the exhibit 
starts in January and slays open through 
February. March, and Xpnl Finally, wc 
read the small print when during the 
week the exhibit is open and how much 
costs. 

The numbers and words “10 - 3 Vlon 
- Fri.’’ mean that the exhibit opens ai 
10:00 in the morning and stays open all 
Hie way until 5:00 at night, on Monday 
through Friday, that means the days 
Monday. I uesday, Wednesday. Thursday . 
and Friday. 

When wc go lo an exhibit, wc might 
see other p oste rs or signs inside the 
building The exhibit will he in the 
direction of the signs. IF wc aren't sure 



Cedarville’s 
Land and Sea Museum 

presents Animal* of It** Arctic 

i April aotli 


Figure 2. Revised text used in the passage. 



Interviews conducted with those learners at the end of the 
session revealed that they did not know that “Mon.-Fri.” meant 
“Monday through Friday” or that it included the days Monday, 
Tuesday, Wednesday, Thursday, and Friday. This was true even 
for some learners who read “Mon.-Fri.” as “Monday through 
Friday.” In this case, learners were answering the question in¬ 
correctly due to an apparent lack of overlap between their exist¬ 
ing verbal repertoires and the stimuli supplied in the poster. We 
then modified the textual stimuli in the passage, as illustrated 
in Figure 2, and correct responding to the questions increased, 
as shown in Figure 3. The modifications to the text in the pas¬ 
sage were designed to increase the overlap between the learn¬ 
ers’ intraverbal repertoires and the contents of the poster. After 
these modifications were implemented, 4 out of 5 naive learners 
in our laboratory responded correctly to the same questions to 
which previously 4 out of 5 learners had responded incorrectly. 

Figure 3 shows a sample of the data collected in our labora¬ 
tory for this segment. The data collection and display is based 
upon a control/analysis programing model first described by 
Sidman and Stoddard (1967). Columns “a” through “f” repre¬ 
sent different passage-question-answer sets. The sample in col¬ 
umn “a” is the set under discussion. A horizontal line indicates 
a response request. An initial uptick indicates the question was 
answered correctly on the first attempt. A down tick indicates 
an incorrect response, which occasioned a correction procedure 
(horizontal line). A subsequent incorrect response produces a 
down tick and occasioned another correction procedure. A cor¬ 
rect response to the correction produces another set, column 
“b.” Initials in each row indicate different learners. The data are 
from the initial five learners, tested before the described change 
was made, and the five learners tested after the change in the 
program. 

The revised version of the passage (in Figure 2) overlaps with 
learners’ verbal repertoires and extends the instructional guid¬ 
ance exerted by the stimuli “Monday, Tuesday, Wednesday, 
Thursday, and Friday” to the stimulus “Mon.-Fri.” The autoclitic 
frames added to the text are ones that already exist in learners’ 
repertoires and possibly already control other, similar respons¬ 
es. The importance of overlapping writer and learner verbal rep¬ 
ertoires cannot be overstated, but it is only part of what it takes 
to teach reading comprehension. 

Another part of this overlap between the learners’ verbal rep¬ 
ertoires and the text is afforded by what is commonly referred to 


as the learners’ vocabulary (Layng et al., 2011; Sota et al., 2011). 
The literature on reading comprehension has identified building 
a core vocabulary as essential for reading comprehension (e.g., 
Tannenbaum, Torgesen, & Wagner, 2006). However, designing 
programs that effectively and efficiently build vocabulary can 
be challenging. Several issues seem to be at the center of this 
challenge. 

First, what is meant by vocabulary? “Vocabulary” does not 
simply refer to words or even to words and their definitions. 
When we speak of vocabulary, we are typically referring to 
words in combination with the many relations those words en¬ 
ter into. These relations can be between the vocabulary word 
and other words (intraverbal relations) and/or nonverbal stim¬ 
uli (tactual relations). An effective vocabulary program, there¬ 
fore, must be designed specifically to establish these relations, 
with instructional strategies and methods chosen based on the 
type of relation to be established. 

Second, how can vocabulary be taught efficiently? Some es¬ 
timates of vocabulary learning assert that average elementary 
school students learn approximately 12 new words per day 
(Anglin, 1993, cited in Bloom, 2002), but programs that explic¬ 
itly teach vocabulary may do so relatively slowly. For example, 
McKeown, Beck, Omanson, and Perfetti (1983) estimate that 
it typically takes about 20 minutes of instruction to explicitly 
teach a single new vocabulary word. 

EXPLICIT VOCABULARY INSTRUCTION 

In programing vocabulary instruction in Headsprout Reading 
Comprehension , we focused on two central issues: (1) to effec¬ 
tively teach vocabulary by teaching the relations that the target 
words enter into and (2) to do so as efficiently as possible. The 
instructional sequence that we programed to teach vocabulary 
words was based on procedures commonly used to teach stimu¬ 
lus equivalence, but with some important differences. Our ob¬ 
jective was not to demonstrate that novel, arbitrarily selected 
stimuli can become members of the same class through expo¬ 
sure to matching-to-sample contingencies. Instead, the goal 
was twofold. First, we wanted to quickly establish a vocabulary 
word, its definition, and a corresponding picture as members of 
the same stimulus class. Second, we wanted to use the definition 
of each word as the stimulus in the class that would “bridge” the 
stimulus class with the learners’ existing verbal repertoires— 
therefore rapidly expanding a learner’s verbal repertoire by 
taking advantage of the learner’s existing repertoire. Below, we 
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Figure 3. Sample of data collected in our laboratory. 

describe the program built to establish 24 relations (four vo¬ 
cabulary words with six relations each) in approximately five 
minutes of interactive instruction. 

Simultaneous presentation and linkage by autoclitic frames. In an intro¬ 
ductory sequence, learners are presented with the target vocab¬ 
ulary word, to which they have to emit an observing response 
(clicking on the word). After the click, learners hear a definition 
of the word, hear a sentence that uses the word, and then see 
a picture illustrating the word. This sequence of events is pre¬ 
sented in Figure 4. 

As shown in Figure 4, we depart from typical stimulus-equiv¬ 
alence approaches by providing learners with the three nodes of 
the stimulus class from the outset. That is, the program attempts 
to establish instructional (S D i) guidance (see Layng et al., 2011) 
of the words, definitions, and pictures from the outset by pre¬ 
senting the stimuli simultaneously and using autoclitic frames 


Word is shown on screen and read aloud by program; e.g. 
“Here’s a word: ‘distant.’ Say distant and click on it.” 



Learner clicks on word 



Definition of the word is given by the program; e.g., “When 
something is distant it is far away.” 



Picture and sentence illustrating the word are shown on screen; 
e.g., “Pip saw that the boat was distant. Now you read the 
sentence. When you’re done reading, click on the picture.” 



Figure 4. Instructional sequence to introduce vocabulary words. 


(Skinner, 1957) to establish the relations between them. In typi¬ 
cal stimulus-equivalence procedures, learners are not exposed 
to the programed stimulus classes in advance, but instead must 
go through a sequence of multiple trials in order to learn those 
classes through direct exposure to the contingencies. Because 
our goal was to establish these stimulus classes as quickly and 
efficiently as possible (as opposed to demonstrating the prop¬ 
erties of typical matching-to-sample training), we were free to 
provide the three nodes of the relation through simultaneous 
presentation and linkage by autoclitic frames. The procedure 
used could be considered one of one-to-many or sample-as- 
node, where all the comparison stimuli that are part of the class 
are associated by virtue of their relation to a single sample. We 
repeat this process for two additional words in order to create 
the first group of three stimulus classes. 

Stimulus equivalence. Once the three words have been introduced, 
the words, the three definitions, and the three pictures are ar¬ 
ranged in a typical stimulus-equivalence paradigm where learn¬ 
ers see a sample and an array of three comparisons and must 
select the comparison that belongs in the same stimulus class as 
the sample. Figure 5 shows an example of this procedure. 

Another departure from the typical stimulus-equivalence 
paradigm is illustrated by the potential familiarity to learners 
of the stimuli used. In typical stimulus-equivalence procedures, 
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Figure 5. Example of stimulus-equivalence procedure. The definition is presented as 
the sample, and three pictures are presented as the comparisons. 


investigators take pains to ensure that the stimuli used in the 
classes are novel to learners (Sidman, 1994). In our program, 
such concern with novelty is nonexistent. The pictures used to 
illustrate the words are novel to the learners, but they represent 
objects that are likely to be familiar to most learners. The defini¬ 
tions for each word were written to maximize the likelihood that 
they would overlap with the learners’ existing verbal repertoires. 
We avoided definitions from dictionaries because they seldom 
allow for such overlap, especially when it comes to learners in 
elementary grades. We created our definitions by first carefully 
examining the different contexts in which each target word is 
likely to appear and by examining the word’s most common 
meanings and usages, and then writing a definition consistent 
with such examination. Thus, the vocabulary words themselves 
may be the only stimuli in the classes that are novel to learners. 
Fast mapping with contingency adduction. While learners are engaged 
in this matching-to-sample activity, the program introduces a 
fourth (not previously seen) word that learners have to match 
to a fourth (not previously seen) definition and then to a fourth 
(not previously seen) picture. In our learning laboratory, the in¬ 
troduction of these novel stimuli resulted in learners responding 
correctly to the new word introduced, almost without excep¬ 
tion. The practice of embedding trials showing a novel sample 
and a novel comparison stimulus in matching-to-sample tasks, 
while keeping the other comparison stimuli as before, is some¬ 
times referred to as “exclusion learning” or “fast mapping” in 
the cognitive psychology literature (see Bloom, 2002, for a dis¬ 
cussion of fast mapping). “Exclusion learning” refers to the no¬ 
tion that when the novel sample and novel comparison are pre¬ 
sented, learners select the novel comparison because all other 
comparisons are already matched to their respective samples. 
Learners, then, are thought to select the correct comparison by a 
process similar to that of elimination, i.e., by exclusion. The label 
“fast mapping” suggests that learners are mapping the relations 
among stimuli. Both labels point to what the programer designs 
rather than to what variables can account for learners’ behavior. 
The programer knows which stimuli to put in the “excluded” 
class that is introduced after the original classes are formed, as 
well as how the new stimuli are “mapped” within a class. 


We approach this somewhat differently. Learners are naive 
with regard to which stimuli are in the excluded class or are to 
be fast mapped. Their behavior is under the instructional (S D i) 
guidance of selecting the comparison stimulus that goes with 
the sample and under the dimensional (S D d) guidance of the 
specific sample-comparisons array presented in each trial. Re¬ 
sponding in accordance with the three stimulus classes formed 
(the first three words and their respective definitions and pic¬ 
tures) is potentiated by the initial instruction and the subse¬ 
quent contingencies set up by the matching-to-sample trials. 
In each trial after the initial training, response alternatives are 
effectually reduced to one sample among the three in the ar¬ 
ray because of previous reinforcement and instruction for re¬ 
sponding within the class. When a novel sample and a novel 
comparison are presented, learners consistently select the “cor¬ 
rect” comparison (the one associated with the new sample) be¬ 
cause all the remaining samples present have been previously 
included in their respective stimulus classes. The new compari¬ 
son is the only one that has not been previously matched with 
a sample. Thus, response alternatives are effectually reduced to 
the new comparison. Dimensional guidance is varied because 
of the presentation of a novel stimulus, but abstractional control 
(selecting the comparison stimulus that goes with the sample) is 
adduced (after Andronis, Layng, & Goldiamond, 1997; Layng, 
Twyman, & Stikeleather, 2004) and maintained. 

The purpose of using this particular matching-to-sample 
procedure (in which we introduce a whole new class of stimuli 
without previous instruction and learners’ responses consistent¬ 
ly conform to the new stimulus class) is efficiency of instruction. 
Once this instructional sequence ends, learners have practiced 
each of the relations between words, definitions, and pictures, 
in all possible directions, for four vocabulary words. This entire 
instructional sequence takes approximately five minutes. As of 
this writing, our online data from over 35,000 learners indicate 
that the learners achieve about 96% correct throughout the in¬ 
structional segment. 

Extension Across Contexts. This activity, however, is not the com¬ 
plete extent of their practice with vocabulary words. Proper S D i 
guidance must occur across a range of stimuli before learners 
can be considered to “truly know” the meaning of a word. That 
is, learners must tact the relation between a word and its mean¬ 
ing in novel verbal contexts different from those in which the 
relation was originally trained —that is, different from the stim¬ 
ulus-equivalence procedure described above. Thus, vocabulary 
words taught through this stimulus-equivalence procedure ap¬ 
pear again in subsequent passages that learners must read and 
about which they must answer reading comprehension ques¬ 
tions. Often, learners must tact the relation between the word 
and its meaning in a novel context provided by different stimu¬ 
lus prompts and by variation of the variable attributes of the 
word meaning, if they are to correctly answer the questions. 

■ PROGRAMING AN INVESTIGATIVE REPERTOIRE 

Because it is impossible to teach each reading comprehension 
question that learners will encounter, it becomes necessary to 
teach a strategy that can be transferred across a range of reading 
comprehension questions. A strategy is a generative repertoire 
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whereby learners engage in supplementary verbal stimulation 
to solve a particular problem (Layng, 2005; Palmer, 1991; Rob¬ 
bins, 2004, 2011). Strategies are defined as a systematic attack 
on problems for which algorithmic (successive conditional dis¬ 
criminations) and principle application are insufficient to result 
in a response that would be reinforced (Tiemann & Markle, 
1990). Strategies require the production, on the individual’s 
part, of supplementary verbal stimulation that potentiates re¬ 
sponding consistent with the S D i guidance exerted by the con¬ 
tingencies implied in the problem at hand (Layng et al., 2011). 
Furthermore, a strategy requires application to novel conditions 
in order to be considered as such. 

As noted by Sota et al. (2011), reading comprehension is not 
a monolithic skill; it rests on different strategies that are rele¬ 
vant to different reading comprehension questions. We prefer 
the term “investigative repertoire” (Layng et al., 2011) to refer 
to the overarching repertoire of inspecting the contingency re¬ 
quirements (see Skinner, 1966) of any reading comprehension 
question presented, be it a literal, inferential, summative, or 
derived-meaning question. Based on an inspection of the con¬ 
tingencies, learners can then apply the strategy that is relevant 
to the specific question at hand (see Robbins, 2004). 

EXPANDING A VERBAL REPERTOIRE THROUGH BUILDING A 
DERIVED-MEANING INVESTIGATIVE REPERTOIRE 

Verbal and investigative repertoires overlap; they are not com¬ 
pletely separate. One place where they overlap is in derived- 
meaning (vocabulary) comprehension. As noted elsewhere 
(Layng et al., 2011; Sota et al., 2011), vocabulary is critical to a 
verbal repertoire. Though some vocabulary is directly taught, 
many words are learned through reading text (Bloom, 2002). 
Accordingly, a strategy for acquiring this type of vocabulary is 
critical to extending a learner’s verbal repertoire. 

In Headsprout Reading Comprehension , the strategy that 
learners are taught for determining the meaning of a word in 
context begins by having learners replace the unknown word 
with the first possible answer, read the sentence formed thereby, 
and then repeat this same task with the remaining possible an¬ 
swers. Take the following as an example: 

Once there were two countries that did not like each other. There was 
so much strife between them that they were always at war. Finally, 
most of the people decided to try to make peace. 

What does “strife” most likely mean? 

O fighting 
O peace 
O space 

Learners are taught to replace the word “strife” with each of the 
possible answers, as follows: 

• There was so much fighting between them that they were al¬ 
ways at war. 

• There was so much peace between them that they were always 
at war. 

• There was so much space between them that they were always 
at war. 

The next step in the strategy is to evaluate which of the three 


Chris cm pull himself up into a tree because he is 
very tird Me can do it with just his arms! He always 
climbs down carefully, though. 

H hat does Jird maxi likely mean? 

O short 

O hungry 
o strong 

Figure 6. Example of question asking learners to derive the meaning of a word from 
context. 

sentences above makes the most sense. For many of our target 
learners, however, the responses required by such instruction 
may not be in place. For example, what does “makes the most 
sense” mean in this context? As noted above, tacting which sen¬ 
tence makes the most sense equates to tacting which string of 
verbal stimuli includes thematic categories that are mutually 
congruent. For example, “peace” and “war” and “space” and 
“war” (the categories present in the second and third sentences 
above) create thematic discrepancies that are readily apparent. 
On the other hand, “fighting” and “war” are in the same the¬ 
matic category. The S D i guidance exerted by “war” is similar to 
that exerted by “fighting,” given the relational autoclitics of the 
sentence. Tacting thematic congruencies and discrepancies is 
not simple to teach, given the wide variety of derived-mean¬ 
ing questions that learners will encounter within the program 
and outside of it (e.g., in reading comprehension exercises or 
tests in the classroom). Thus, we took an “EGRUL” approach to 
this task. EGRUL approaches use different examples and non¬ 
examples in order to guide learners to “discovering” the rule, as 
opposed to “RULEG” approaches, in which the rule is provid¬ 
ed first and followed by examples and non-examples (Markle, 
1969; 1981; 1991). 

For this instructional sequence, the program goes through 
several sentences that either “make sense” or don’t, thus demon¬ 
strating what the term means via carefully designed examples 
and non-examples. Specifically, learners first read the passage, 
question, and possible answers, examples of which are present¬ 
ed in Figure 6. Note that the target word is a non-word. This 
deliberate choice forces learners to use the process taught rather 
than rely on already established occasion-behavior pairs (see 
Sota et al., 2011). 

After reading the content shown in Figure 6, learners hear, 
“Let’s see how each answer fits in the sentence.” The program 
then substitutes each possible answer for the target word in the 
passage, one at a time, as shown in Figure 7. 

While learners see the content shown in Figure 7, the narra¬ 
tor reads aloud the sentences formed by inserting each possible 
answer. After each sentence is read, learners hear either “That 
makes sense!” or “That doesn’t make the most sense here.” For 
the top panel of the figure above, learners would hear, ‘“Chris 
can pull himself up into a tree because he is very short. He can 
do it with just his arms!’ That doesn’t make the most sense here.” 
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Chris can pull himself up into a tree because he is 
very short He can do it with just his arms! 

What does fird mast likely mean? 

o short 
O hungry 

O s<n>ng 

Chris can pull himself up into a tree because he is 
very hungry . He can do ii with just his amis! 

What does fird mast likely mean ? 

O ^hort 

O hungry 

O strong 

Chris can pull himself up into a tree because he is 
very strong . He can do it with just his arms! 

What does fird mast likely mean? 

O 

Q hungry 

O *<rong 

Figure 7. Example of sequence to teach learners to derive the meaning of a word 
from context. 

The verbal stimuli from the screen and the audio are designed 
to overlap with learners verbal repertoires. After one iteration 
of this sequence, the program no longer evaluates whether each 
sentence formed makes sense. That is left to learners, with the 
narrator asking, “Does that make sense?” after each sentence is 
read. This sequence repeats with a range of questions that ask 
for the meaning of a variety of words (nonsense words, as well 
as actual words that include nouns, adjectives, verbs, and ad¬ 
verbs) across a variety of passages. Learners are thus taught an 
investigative repertoire to apply across questions related to de¬ 
riving the meaning of words from context. 

The description above is a summarized account of the inves¬ 
tigative repertoire put in place by the program to teach how to 
derive the meaning of words from context. Similar investigative 
repertoires are programed for literal, inferential, and summa- 
tive reading comprehension, but space precludes a detailed ac¬ 
count of each. The next section focuses on the programing of 
an investigative repertoire to answer inferential comprehension 
questions. 

PROGRAMING AN INFERENTIAL INVESTIGATIVE REPERTOIRE 

Inferential reading comprehension can be defined as the pro¬ 


cess of answering a question that is related to a passage, but the 
answer to which does not have a one-to-one topographic cor¬ 
respondence with the passage (Sota et al., 2011). The learner, 
for example, may be asked to predict, based on current infor¬ 
mation, a future event in a story, to estimate how a character is 
feeling, or to deduce the probable cause of an event, to name a 
few. How does one program a repertoire that encompasses such 
seemingly disconnected responses? 

As with other complex repertoires, there may be multiple 
ways to program for the same terminal repertoires. Below, we 
describe one program that has proven effective in our learning 
laboratory. 

Our first step in designing a strategy for answering inferential 
comprehension questions was to analyze the concept of infer¬ 
ential questions (see Sota et al., 2011). Questions that are desig¬ 
nated as inferential have the following critical attributes: 

1. The answer category appears in the passage. 

2. The answer meets the criteria specified in the question. 

3. The answer does not have a topographic correspondence 
with the words in the passage. 

This concept analysis of inferential comprehension simplifies 
programing in several ways. First, it makes the programing of 
coordinated concepts clearer and more efficient. In the case of 
an inferential question, its coordinate concept is a factual or lit¬ 
eral question, described by Sota et al. (2011). Literal and infer¬ 
ential questions are coordinate concepts. That is, they only differ 
in the specific value of one of their critical attributes. While fac¬ 
tual questions require topographic correspondence between the 
answer and the words in the passage, inferential questions re¬ 
quire the lack of such topographic correspondence. This sharing 
of all but one critical attribute allows programers to find com¬ 
monalities in the strategies and leverage those commonalities to 
minimize the number of “rules” that learners have to remember. 

The second contribution of a concept analysis to program¬ 
ing lies in its delineation of what constitutes a correct answer 
to an inferential comprehension question. The characteristics of 
a correct answer (i.e., one that has the three critical attributes 
outlined above) can then be taught to learners explicitly. We will 
describe how we programed for this attribute-based instruc¬ 
tional system in greater detail below. 

Third, the concept analysis sets the ground for programs that 
facilitate transfer across a wide variety of stimuli (in this case, a 
wide range of inferential questions). The concept analysis indi¬ 
cates the range of variable attributes that must be systematically 
varied across inferential comprehension questions in order to 
vary the dimensional guidance or S D d, in this case represented 
by the particular question and passage characteristics, while 
keeping the instructional guidance or S D i constant (see Layng 
et al., 2011). 

The first critical attribute (the answer must appear in the pas¬ 
sage) was addressed by inserting a step designed to evoke re¬ 
sponses that increase the likelihood that learners responses are 
guided by the passage. The program directs learners to “click 
the part of the passage that helps [them] answer the question” 
before they can select an answer. By attending to and actively re¬ 
sponding to the portion of the passage that is thematically relat¬ 
ed to the question, that is, matching S D i guidance, the likelihood 
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Tnsh and filakc arc resting h\ the lake. 
Tmh thinks ahnit thing* she hopes lor. 

She imagines a prctt> new hike. She 
could go on hike rides with her friends' 
She also hopes lor a stuns ring 
Sudden Is. Hlake says. “What a 
wonderful day!" He smiles a hie smile 
Tnsh looks uround. "Yeah'" she says 
"We ha\c e\ cry thing we need. It s fun 
to wish, hut I feel happy with just what 
I ha\e!" 


Figure 8. Example of sequence to teach learners to look for the answer in the passage 

before answering the question, 

of a correct response is further increased (Layng et al., 2011). 
An example of what learners see during this sequence is shown 
in Figure 8. Learners first read the passage, question, and pos¬ 
sible answers, but cannot select an answer at this point. Instead, 
they are directed to click on the part of the passage that will 
help them answer the question. The passage comprises several 
clickable spots, and learners receive confirmatory or corrective 
feedback depending on which part of the passage they click (in 
the figure above, the correct part of the passage is “She imagines 
a pretty new bike”). After selecting the part of the passage that 
helps them answer the question, learners move on to selecting 
one of the three possible answers presented. 

The second critical attribute of inferential questions (the an¬ 
swer meets the criteria specified in the question) is shared by 
at least two types of comprehension questions (literal and in¬ 
ferential), and thus is taught as a skill common to both types 
of questions. Figure 9 shows an example of what learners see 
while going through this prerequisite skill instruction. Learners 
are presented with short passages followed by a question about 
the passage. Three different sections of the passage (sentences 
or phrases) are underlined in different colors to potentiate dis¬ 
crimination. Learners have to click on the underlined part that 
shows the answer to the question. The questions presented vary 
systematically in the type of characteristic asked about and cov¬ 
er the question words when , where , how, who , what, and why. As 
mentioned above, learners are required only to click on the part 
of the passage that contains the answer to the question, not yet 
to answer the question in its entirety. The purpose of this prepa¬ 
ratory instruction is to provide learners with a uniform reper¬ 
toire of tacting the contingency requirements of each question 
based on the question words. 

The third critical attribute (the answer does not have a topo¬ 
graphic correspondence with the words in the passage) dif¬ 
ferentiates inferential comprehension from literal comprehen¬ 
sion. If the answer to the question does not have a topographic 
correspondence with words in the passage, then topographic 
correspondence cannot be a source of stimulus control when 
selecting the correct answer. Learners behavior when answer¬ 
ing inferential questions must rely on thematic or category cor¬ 
respondences present in the passage, question, and answers. 
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Figure 9. Example of instruction common to literal and inferential comprehension, 


Skinner (1957) used the term thematic prompt to refer to a 
verbal stimulus that increases the likelihood of a target re¬ 
sponse. Key words in the passage can be considered thematic 
prompts inasmuch as they are instructional stimuli (S D i) that 
restrict response alternatives to the category that matches the 
instructional stimuli in the inferential question (see Layng et al., 
2011). The problem is that often these thematic prompts are in¬ 
effective at evoking the target response by their mere presence. 
The task, then, is to get learners to respond to those thematic 
elements. 

What do people who are adept at inferential comprehension 
do in order to find thematic correspondences between words 
in the passage and words in one of the possible answers (the 
“correct” answer)? According to cognitive psychology, they ac¬ 
tivate and use their background knowledge. This explanation, 
however, is insufficient to design a program that systematically 
and explicitly teaches learners to make inferences; it does not 
provide any indication of the operations that need to take place 
in order to activate background knowledge. For example, what 
are learners actually doing when they are “activating their back¬ 
ground knowledge?” And how can a program explicitly and sys¬ 
tematically teach that behavior? 

When learners are drawing on their background knowledge, 
they are engaging in intraverbal responses that are thematically 
related to the question at hand. If we want to teach learners to 
systematically produce intraverbal responses that are themati¬ 
cally related to the question, then we have to provide explicit 
instruction on this sort of response generation. The reader will 
recall the steps that Jeffrey, a child in our early example, takes to 
find the inferred answer. When he cannot find the exact answer 
in the passage, he says, “I’ll have to look for clue words that help 
me think about how Sam was feeling.” But to teach that behavior, 
we need to establish the repertoire described by “think about,” 
as learners cannot simply be asked to “think about.” 

Instead, they are provided with examples of what they are 
doing when they are “thinking about” something. We begin by 
having the learners read the question about the passage. This 
is critical, since the criteria for selecting the correct answer are 
specified by the question, and the answer must fall in the same 
thematic category as a category that appears in the passage. 
Then, the narrator rereads the question and tells the learner, 
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cookie 

happy 

good 

Gus 

moon 

glad 

rock 

goofy 


Figure 10. Example of early sequence designed to activate background knowledge. 

“We need to look for clue words that make us think about_.” 

For example, if the question is asking about how a character 
feels, then learners are told, “We need to look for clue words 
that make us think about feelings.” Here, again, we designed an 
EGRUL approach in which learners are guided through a se¬ 
ries of examples and non-examples (Tiemann & Markle, 1990). 
Learners select from a list of items that may make the learner 
think about what the question is asking (for example, items that 
may make the learner think about feelings). Figure 10 shows a 
list of examples and non-examples of items that make learners 
think about feelings. 

When presented with a frame like the one in Figure 10, learn¬ 
ers hear the instruction “Click on the words that make you 
think about feelings” and have to select all the examples among 
an array of examples and non-examples. Confirmatory or cor¬ 
rective feedback follows each learner response. The objective 
of this step is restriction of response alternatives to those that 
are thematically related to the question (in this case, a question 
about feelings). The responses evoked during this instructional 
sequence do not need to be exhaustive. They simply need to 
be in the same thematic category as the target response. When 
these responses are made more likely, the probability of a cor¬ 
rect answer to the question is also made more likely, because the 
correct answer is in the same class. 

Next, we have learners go back to the passage and examine a 
selected set of phrases underlined in different colors (see Figure 
11 for an example). Learners have to find clue words that fall in 
the same thematic category as the items they just selected when 
they were “thinking about.” 

Now that responses thematically related to the question have 
been made more likely, three possible answers are presented, 
and the learner is instructed to select the one that answers the 
question (see Figure 12). Typically, one of the possible answers 
meets all of the critical attributes listed above except for the first 
one. Another possible answer meets all of the critical attributes 
except for the second one. The third possible answer meets all 
of the critical attributes. If learners select an incorrect answer 
(one lacking one critical attribute), the program provides spe¬ 
cific feedback about the attribute that is missing. 

Naturally, the extensive support involved in presenting lists 
of examples and non-examples for each question cannot and 


f u'wwWAtwwwwwwjV^/wik w 

(ius knew jTc^asJjcinu^sill^. Still, cvcrv^inicjicjookcd at 
the moon, he saw a bi g coo kie. 

How did Gus feci about himself w hen he looked at the moon ? 


* * 

Figure 11. Example of frame where learners find clue words in the passage. 

should not be maintained throughout, as the ultimate goal is 
that learners will be able to do this independently. After the cri¬ 
terion is met for this stage of instruction, the support is with¬ 
drawn and learners are encouraged to “think about clue words” 
independently, without the program supplying examples or 
non-examples. This encouragement is later removed completely. 

Earlier in this discussion, we mentioned that learners can be 
taught explicitly to respond across the critical attributes of an 
inferential question (i.e., the answer category appears in the pas¬ 
sage, although the answer does not have a point-to-point topo¬ 
graphic correspondence with it, and it meets the criteria speci¬ 
fied in the question). This is done partially by restricting 
response alternatives that are already at some strength in learn¬ 
ers’ repertoires in order to make the target response more likely 
(see Goldiamond & Thompson, 1967/2004). Lets consider one 
example. 


The school bus pulled up to the bus stop. It waited there for five 
minutes. Finally, all the children came and got on. Then the school bus 
went on down the street. 

Why did the school bus wait at the bus stop? 

O It waited there for five minutes. 

O The bus driver was late. 

O Some children were late. 

Unless response alternatives are restricted, learners’ respond¬ 
ing is under the guidance of competing S D i. On one hand, their 
existing repertoires may make the response “the bus driver was 
late” likely, because that is a valid reason for a bus to have to 
wait. This response is probably at some strength for many learn¬ 
ers without their needing to attend to the textual stimuli in the 
passage. For each question presented in the program, there is 
at least one possible answer that is a likely response if one con¬ 
siders exclusively the learners’ verbal repertoires. We selected 
these possible answers, or distracters, by design, in order to 
teach learners to use the strategies taught in the program and 
not rely exclusively on their past verbal repertoires. The other 
distracter, “it waited there for five minutes,” is probably unlikely 
to exist in learners’ intraverbal repertoires in any meaningful 
strength before this exercise. The passage, however, contains the 
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Gus knew he was being silly. Still, every time he looked at 
the moon, he saw a big cookie. 

flow did Gus feel about himself when he looked at the moon? 
O a round 

O a little funny 

O a little short 


Figure 12. Example of frame where learners select answer after having identified clue 

words in the passage, 

words “it waited there for five minutes,” and that response may 
momentarily increase in probability because of the textual and 
echoic components that evoke it. Again, this type of distracter (a 
possible answer with words that appear in the passage but that 
do not answer the question) is used frequently in the program 
by design. 

When learners select one of these distracters, the feedback 
they receive indicates why the distracter is not a correct an¬ 
swer. For example, the programs auditory feedback may inform 
learners that “we see some of those words in the passage, but 
they don’t answer the question.” Similar feedback, specific to the 
critical attribute missed, is provided if learners select the dis¬ 
tracter that is strong in their existing repertoires but that does 
not have a thematic correspondence with the textual stimuli in 
the passage. Thus, this sequence shapes learners’ responding 
along the critical attributes of inferential questions. This strat¬ 
egy is used throughout the program, with some modifications 
as the program advances in order to withdraw program support 
and encourage learner independence. What varies throughout 
the program is the passages from which questions are drawn, 
the reading level of the passages, and each of the variable at¬ 
tributes of the questions, as in the concept analysis described by 
Sota et al. (2011). 

PROGRAMING A GENERAL INVESTIGATIVE REPERTOIRE 

As noted above, the program teaches specific strategies, or in¬ 
vestigative repertoires, for each comprehension type. Each use 
of a particular strategy is in turn occasioned by a more general 
strategy that helps the learner identify which specific strategy 
is required. To accomplish this, the learner must categorize the 
question with regard to the type of comprehension strategy 
that will be required to answer it. To accomplish this, an icon 
is paired with each specific strategy. The icons are depicted in 
Figure 13. 

When presented with a passage-question-answers set, the 
learner is taught to select an icon that represents the strategy 
required by the question. A “fish race” game is used to prac¬ 
tice identifying the strategy needed to answer a question. Dis¬ 
tinguishing between literal and inferential questions, however, 
requires an examination of the possible answers in relation to 
the passage to determine if a literal match or a categorization 



Figure 13. A specific icon is paired with each specific type of comprehension strategy, 

match is required. 

When first presented with a passage that has multiple ques- 
tion-and-answer sets, the learner is required to first read the 
passage, then read the question, and then click on the icon rep¬ 
resenting what the question is asking the learner to do. This in¬ 
creases the likelihood that the learner will ask him - or herself, 
“What is this question asking me to do?” Next, the learner reads 
the answers and is required to click on the area of the passage 
that answers the question. The learner then selects the answer to 
the question using the strategy particular to that question type. 
Over the course of many lessons, the individual steps are re¬ 
moved until the learner completes the sequence independently. 

Space precludes a discussion of all the steps that are required 
to establish this repertoire, but the terminal repertoire is de¬ 
scribed for Jeffrey at the beginning of this article and follows 
this sequence: 

Step 1. Determine what the question is asking. In this case, it is 
either a Find the Fact or a Clue Words question. First, 
Jeffrey reads the passage, the question, and the possible 
answers. Next, he determines what the question is ask¬ 
ing him to do. 

Step 2. The learner determines if there is a 1:1 match between 
the possible answers and the text, which would make 
this a Find the Fact question. “It wants me to find how 
Sam feels. I’ll look back in the passage to see if I can find 
sad,’ ‘happy,’ or ‘funny.’” 

Step 3. The learner does not find a 1:1 correspondence, which 
occasions the Clue Words strategy. “Then he looks back 
in the passage: ‘No... no... I can’t find any of those 
words in the passage.’” (If there were a 1:1 correspon¬ 
dence, the learner would use the Find the Fact strategy 
to select the answer that matched the words found in 
the passage.) 

Step 4. The learner begins to use the strategy specific to an¬ 
swering inferential questions. “I’ll have to look for clue 
words that help me think about how Sam was feeling.” 
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Figure 14. The “fish race” game. Learners must click on the icon that represents the 
type of comprehension strategy reguired. For the passage-guestion-answer set 
depicted, the learner must see that there is no 1:1 correspondence between the 
words in the answer and the words in the passage and, therefore, choose the Clue 
Words icon. 

■ CONCLUSION _ 

What we have described here is portions of a program that 
teaches learners how to comprehend text. Headsprout Reading 
Comprehension is a product of a concerted team effort that in¬ 
volved graphic artists, animators, audio engineers, software en¬ 
gineers, database specialists, network engineers, user test spe¬ 
cialists, program editors, instructional designers and learning 
scientists. It took over three years and more than $2 million to 
produce. We began with the foundation described by Layng et 
al. (2011), performed the analyses detailed by Sota et al. (2011), 
and through continuous testing, revising, and retesting pro¬ 
duced the program which is the topic of this paper. 

Other programs may be developed that achieve similar out¬ 
comes through different procedures. Nevertheless, insofar as 
the program described here is being used successfully to teach 
learners how to comprehend text, we have at least a reasonable 
basis to conclude that the thinking and analytical skills involved 
in comprehension can be explicitly taught (Layng et al., 2011; 
Sota et al., 2011). Furthermore, the consistency in applying 
the complex patterns taught in our program by more than 150 
individual learners who participated in developmental testing 
and more than 35,000 school users leaves us encouraged that 
we accomplished what we set out to do. It has not escaped our 
attention that this consistent performance—in answering ques¬ 
tions correctly and in what learners seem to be “thinking” while 
doing so—is a function of our program, and that programs such 
as ours may provide a framework for effectively teaching think¬ 
ing of all kinds. 
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